Evaluation of Ranking Similarity in Ordinal Ranking Problems

In ordinal ranking problems objects, alternatives, products, services, etc. are ranked by several experts and the goal is to convert a set of (generally different) rankings into the final group consensus ranking. However, this goal depends on a degree of agreement among rankings. With random rankings one cannot expect to get meaningful consensus, but if rankings are „close“ and represent agreement between experts, then the final group consensus has much more sense. The aim of this article is to present the evaluation of similarity among rankings, which is based on Kendall’s τ and W, Spearman’s ρ, Pearson’s r and dot product of vectors. Cases without and with ties are discussed as well as a problem of similarity between incomplete rankings (top k lists). Explanations are based on examples.